Quickstart

Start the Server

SIE’s primary target is x86 Linux nodes with NVIDIA GPUs. The CPU image lets you try everything locally; for production deployment with autoscaling and multi-GPU, see Deployment.

macOS (Apple Silicon)

Native install, served on Metal, no Docker (requires Python 3.12):

# embeddings + reranking (torch-MPS)
pip install "sie-server[local]" && sie-server serve   # http://localhost:8080

# generation (Apple MLX) runs in its own env; uv makes it a one-liner
# (or pip install "sie-server" "mlx-lm>=0.30.7" into a fresh venv)
uvx --with "mlx-lm>=0.30.7" --from sie-server sie-server serve -b sglang -p 8081

Or run the Linux CPU image under emulation:

docker run --platform linux/amd64 -p 8080:8080 \
  -v sie-hf-cache:/app/.cache/huggingface \
  ghcr.io/superlinked/sie-server:latest-cpu-default

Linux (CPU)

docker run -p 8080:8080 \
  -v sie-hf-cache:/app/.cache/huggingface \
  ghcr.io/superlinked/sie-server:latest-cpu-default

Linux (NVIDIA GPU)

docker run --gpus all -p 8080:8080 \
  -v sie-hf-cache:/app/.cache/huggingface \
  ghcr.io/superlinked/sie-server:latest-cuda12-default

The server starts on port 8080 with all models available. Models load on first request.

Apple Silicon runs natively

pip install "sie-server[local]" serves embeddings and reranking on torch-MPS; generation runs as a separate Apple MLX process (-b sglang selects MLX on macOS). This is the recommended local experience on Apple Silicon. The [local] extra matters: it pins transformers>=4.57,<5, the version the embedding and reranking adapters need; a bare pip install sie-server can resolve transformers 5.x and fail at runtime. The --platform linux/amd64 Docker image still works but runs under QEMU/Rosetta emulation. For production, run on x86 Linux with CUDA.

Kubernetes clusters

If connecting to a Kubernetes cluster with scale-to-zero, first requests may return 503 Service Unavailable with an X-SIE-Error-Code: PROVISIONING header and Retry-After: 60 while workers provision. Use wait_for_capacity=True (Python) or waitForCapacity: true (TypeScript) in the SDK, or expect 3-7 minute cold starts. See Scale-from-Zero.

Install the SDK

Python

pip install sie-sdk

TypeScript

npm install @superlinked/sie-sdk   # pnpm and yarn work too

Generate Embeddings

Python

from sie_sdk import SIEClient
from sie_sdk.types import Item

client = SIEClient("http://localhost:8080")

# Single item
result = client.encode("sentence-transformers/all-MiniLM-L6-v2", Item(text="Hello world"))
print(result["dense"].shape)  # (384,)

# Batch
results = client.encode("sentence-transformers/all-MiniLM-L6-v2", [
    Item(text="First document"),
    Item(text="Second document"),
])
print(len(results))  # 2

TypeScript

import { SIEClient } from "@superlinked/sie-sdk";

const client = new SIEClient("http://localhost:8080");

// Single item
const result = await client.encode("sentence-transformers/all-MiniLM-L6-v2", { text: "Hello world" });
console.log(result.dense?.length);  // 384

// Batch
const results = await client.encode("sentence-transformers/all-MiniLM-L6-v2", [
  { text: "First document" },
  { text: "Second document" },
]);
console.log(results.length);  // 2

The first call to a model downloads its weights from Hugging Face and loads them; all-MiniLM-L6-v2 is ~90MB, so the first result lands in seconds to a couple of minutes, and warm calls typically return in milliseconds. When you want a multilingual flagship embedder with dense and sparse output, swap the model ID to BAAI/bge-m3 (~2.3GB download on first use); the call stays identical.

Rerank Search Results

Python

query = Item(text="What is machine learning?")
items = [
    Item(text="Machine learning uses algorithms to learn from data."),
    Item(text="The weather is sunny today."),
]

result = client.score("cross-encoder/ms-marco-MiniLM-L-6-v2", query, items)

for entry in result["scores"]:
    print(f"Rank {entry['rank']}: score={entry['score']:.3f}")
# Rank 0: score=-7.104
# Rank 1: score=-11.048
# (cross-encoder logits; the relative order is what matters)

TypeScript

const query = { text: "What is machine learning?" };
const items = [
  { text: "Machine learning uses algorithms to learn from data." },
  { text: "The weather is sunny today." },
];

const result = await client.score("cross-encoder/ms-marco-MiniLM-L-6-v2", query, items);

for (const entry of result.scores) {
  console.log(`Rank ${entry.rank}: score=${entry.score.toFixed(3)}`);
}
// Rank 0: score=-7.104
// Rank 1: score=-11.048
// (cross-encoder logits; the relative order is what matters)

Extract Entities

Python

result = client.extract(
    "urchade/gliner_multi-v2.1",
    Item(text="Tim Cook is the CEO of Apple."),
    labels=["person", "organization"]
)

for entity in result["entities"]:
    print(f"{entity['label']}: {entity['text']}")
# person: Tim Cook
# organization: Apple

TypeScript

const result = await client.extract(
  "urchade/gliner_multi-v2.1",
  { text: "Tim Cook is the CEO of Apple." },
  { labels: ["person", "organization"] }
);

for (const entity of result.entities ?? []) {
  console.log(`${entity.label}: ${entity.text}`);
}
// person: Tim Cook
// organization: Apple

Generate Text

Text generation runs on the GPU generation image, not the CPU image above; stop the first server, then start this one on the same port. On Apple Silicon, generation is the separate MLX process from the platform tab above, which serves on port 8081, so create the client against http://localhost:8081 for this step.

docker run --gpus all -p 8080:8080 \
  -v sie-hf-cache:/app/.cache/huggingface \
  ghcr.io/superlinked/sie-server:latest-cuda12-sglang

Python

# On Apple Silicon: client = SIEClient("http://localhost:8081")
result = client.generate(
    "Qwen/Qwen3-0.6B",
    "Reply with a single word: the capital of France.",
    max_new_tokens=16,
    temperature=0.0,
)
print(result["text"])  # Paris

TypeScript

// On Apple Silicon: const client = new SIEClient("http://localhost:8081");
const result = await client.generate(
  "Qwen/Qwen3-0.6B",
  "Reply with a single word: the capital of France.",
  { maxNewTokens: 16, temperature: 0.0 },
);
console.log(result.text);  // Paris

generate is an early surface: the blocking call returns the full result once generation finishes. For streaming and chat-shaped requests, see Text Generation.

What’s Next

• Choosing a Model - pick the right model for your use case
• Dense Embeddings - output types, query vs document encoding
• Text Generation - streaming, sampling options, and chat-shaped requests
• Model Catalog - all 100+ supported models
• Integrations - LangChain, LlamaIndex, Haystack, and more
• Deployment - Docker, Kubernetes, cloud deployment
• Sparse / Hybrid Search - combine dense and sparse for better retrieval